M1 MIDS/MFA/LOGOS |
| Année 2025 |
\[C(X,Y) = \mathbb{E}\left[X Y^\top\right]\]
\[\begin{bmatrix} C_{xx} & C_{xy} \\ C_{xy}^{\top} & C_{yy}\end{bmatrix}\]
The first canonical components are the solution of the next problem
\[\begin{array}{lll}\text{Maximize} & & u^\top C_{xy} v \\\text{subject to} & & u^\top C_{xx} uv=1 =v^\top C_{yy} v \end{array}\]
Let
\[U \times D \times V^\top\]
be a SVD of
\[C_{xx}^{-1/2} \times C_{xy} \times C_{yy}^{-1/2}\]
The solution to the optimization problem above is
\[a = C_{xx}^{-1/2} u_1 \qquad \text{and} \qquad b= S_{yy}^{-1/2} v_1\]
where \(u_1\) and \(v_1\) are the leading left and right singular vectors of \(C_{xx}^{-1/2} \times C_{xy} \times C_{yy}^{-1/2}\), that is the first column vectors of \(U\) and \(V\).
Proof:
A sequence of canonical components of \(C_{xy}\) can be obtained from the sequence of (extended) left and right singular vectors of \(C_{xy}\) with respect to \(C_{xx}\) and \(C_{yy}\)
Proof:
Let \(H_X\) (resp. \(H_Y\)) be orthorgonal projection matrix on the linear space spanned by the columns of \(X\) (resp. \(Y\)).
Canonical correlations \(ρ_1 \geq \ldots \geq \rho_s, \ldots\) are the positive square roots of the eigenvalues \(\lambda_1, \ldots \geq \lambda_s, \ldots\) of \(H_X \times H_Y\) (which are the same as \(H_Y \times H_X\)): \(ρ_s = λ_s\) Vectors \(U^1, \ldots, U^{p_1}\) are the standardized eigenvectors corresponding to the decreasing eigenvalues \(λ_1 \geq \ldots \geq \lambda_{p_1}\) of \(H_X \times H_Y\)
Vectors \(V^1, \ldots, V^{p_2}\) are the standardized eigenvectors corresponding to the decreasing eigenvalues \(λ_1 \geq \ldots \geq \lambda_{p_2}\) of \(H_X \times H_Y\)
R
cancor() from base package R
Function cancor(x, y, xcenter=T, ycenter=T) computes the canonical correlations between two data matrices x and y. Henceforth we assume that the columns of x and y are centered. Matrices x and y have the same number \(n\) of rows. x (resp. y) has p1 (resp. p2) columns.
The canonical correlation analysis seeks linear combinations of the
yvariables which are well explained by linear combinations of thexvariables. The relationship is symmetric as well explained is measured by correlations.
The result is a list of five components
cor correlations.xcoef estimated coefficients for the x variables.ycoef estimated coefficients for the y variables.Our assumption above allows us to assume xcenter and ycenter are zeros.
The next example is taken from the documentation. Use ?LiveCycleSavings to get more information on the dataset.
LifeCycleSavings |>
as_tibble() |>
slice_sample(n=5)# A tibble: 5 × 5
sr pop15 pop75 dpi ddpi
<dbl> <dbl> <dbl> <dbl> <dbl>
1 12.1 23.3 4.41 1508. 3.93
2 12.9 42.2 0.83 728. 4.56
3 18.6 45.2 0.56 138. 5.14
4 12.8 46.3 1.12 152. 2
5 13.2 23.8 4.43 2108. 3.82
Call:
lm(formula = sr ~ pop15 + pop75 + dpi + ddpi, data = LifeCycleSavings)
Residuals:
Min 1Q Median 3Q Max
-8.2422 -2.6857 -0.2488 2.4280 9.7509
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 28.5660865 7.3545161 3.884 0.000334 ***
pop15 -0.4611931 0.1446422 -3.189 0.002603 **
pop75 -1.6914977 1.0835989 -1.561 0.125530
dpi -0.0003369 0.0009311 -0.362 0.719173
ddpi 0.4096949 0.1961971 2.088 0.042471 *
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 3.803 on 45 degrees of freedom
Multiple R-squared: 0.3385, Adjusted R-squared: 0.2797
F-statistic: 5.756 on 4 and 45 DF, p-value: 0.0007904[1] 0.8247966 0.3652762This tells us that highest possible linear correlation beween a linear combination of pop15, pop75 and a linear combination of sr, dpi, ddpi is res.cca$cor[1]. The coefficients of the corresponding linear combinations can be found on the rows of components xcoef and ycoef
Check that the different components of the output of cancor() satisfy all properties they should satisfy.
Design a suite of tests (using testthat) that any contender of the implementation provided by package stats should pass.
TODO
CCA
Abstract of CCA: An R Package to Extend Canonical Correlation Analysis
Canonical correlations analysis (CCA) is an exploratory statistical method to highlight correlations between two data sets acquired on the same experimental units. The
cancor()function inR(RDevelopment Core Team 2007) performs the core of computations but further work was required to provide the user with additional tools to facilitate the interpretation of the results.
As in PCA, CA, MCA, several kinds of graphical representations can be displayed from the results of CCA:
nutrimouse dataset from CCA.nutrimouse in the global environment (see list2env())gene and lipid
Correlation computed with
• Method: 'pearson'
• Missing treated using: 'pairwise.complete.obs'c_XY_long <- c_XY |>
tidyr::pivot_longer(cols=-c(term), names_to="term2", values_to="cor") gene and lipid, save the result in res.cca
sum(res.cca$cor)[1] 21\(H_X\times H_Y\) has 21 eigenvalues equal to \(1\). As the subspaces defined by the columns in gene and lipid have dimensions at most 21 and 40, \(H_X\times H_Y\) equals the projection of \(\mathbb{R}^{40}\) over the smallest subspace.
Sample 10 columns from gene and lipid and repeat the operation
Screeplot
res.cca$cor |>
as_tibble() |>
gt::gt() |>
gt::fmt_scientific() |>
gt::tab_caption("Canonical correlations between `gene` columns of nutrimouse and `lipid` columns")| value |
|---|
| 9.62 × 10−1 |
| 8.82 × 10−1 |
| 7.90 × 10−1 |
| 7.35 × 10−1 |
| 6.96 × 10−1 |
| 5.66 × 10−1 |
| 5.09 × 10−1 |
| 2.67 × 10−1 |
| 1.58 × 10−1 |
| 6.62 × 10−2 |
res.cca$cor |>
as_tibble() |>
mutate(PC=as.factor(1:n), eig=value^2, percent=eig, cumulative=cumsum(eig)) |>
ggplot() +
aes(x=PC, y=eig, label=eig) +
geom_col(fill="white", color="black") +
theme_minimal() +
labs(
title="Squared Canonical Correlations",
subtitle="sample of 10 genes and 10 lipids",
caption="nutrimouse data"
)
Build a correlation circle
We use function plt.cc from CCA.
CCA::plt.cc(
res.cc,
var.label=T,
ind.names = str_c(genotype, diet, sep="-")
)